[SPARK-46581][CORE] Update comment on isZero in AccumulatorV2#44583
Closed
davintjong-db wants to merge 5 commits intoapache:masterfrom
Closed
[SPARK-46581][CORE] Update comment on isZero in AccumulatorV2#44583davintjong-db wants to merge 5 commits intoapache:masterfrom
davintjong-db wants to merge 5 commits intoapache:masterfrom
Conversation
HyukjinKwon
reviewed
Jan 4, 2024
davintjong-db
changed the title
davintjong-db
changed the title
cloud-fan
reviewed
Jan 5, 2024
| * Returns if this accumulator is zero value or not. e.g. for a counter accumulator, 0 is zero | ||
| * value; for a list accumulator, Nil is zero value. | ||
| * Returns false if this accumulator has been updated. Note that this can be true even when | ||
| * the value is a zero value, if the accumulator was updated with a zero value. |
Contributor
There was a problem hiding this comment.
I think the previous comment is fine. Each accumulator can define the semantic of zero value by its own.
Contributor
Author
There was a problem hiding this comment.
That's fair, reverted
cloud-fan
approved these changes
Jan 8, 2024
Contributor
|
The failed pyspark failure is unrelated, merging to master! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
Two changes:
AccumulatorV2'sisZeroto reflect what it actually does.SQLMetricstodefaultValidValueto reflect thisWhy are the changes needed?
AccumulatorV2'sisZerodoesn't do what the comment implies - it actually checks if the accumulator hasn't been updated.The comment implies that for a
LongAccumulator, for example, a value of0would causeisZeroto betrue. But if we were toadd(0), then the value would still be0butisZerowould returnfalse.Changing the name of
zeroValuetodefaultValidValueto avoid confusion sinceisZerodoesn't usezeroValueinSQLMetric.Thanks @arvindsaik for pointing this out.
Does this PR introduce any user-facing change?
No.
How was this patch tested?
Existing tests.
Was this patch authored or co-authored using generative AI tooling?
No.